Characterizing the Solution Path of Multicategory Support Vector Machines
نویسندگان
چکیده
An algorithm for fitting the entire regularization path of the support vector machine (SVM) was recently proposed by Hastie et al. (2004). It allows effective computation of solutions and greatly facilitates the choice of the regularization parameter that balances a trade-off between complexity of a solution and its fit to data. Extending the idea to more general setting of the multiclass case, we characterize the coefficient path of the multicategory SVM via the complementarity conditions for optimality. The extended algorithm provides a computational shortcut to attain the entire spectrum of solutions from the most regularized to the completely overfitted ones.
منابع مشابه
One-against-all multicategory classification via discrete support vector machines
Discrete support vector machines (DSVM), recently proposed in [l01 and [ l l ] for binary classification problems, have been shown to outperform other competing approaches on well-known benchmark datasets. Here we address their extension to multicategory classification, by developing a one-against-all framework in which a set of binary discrimination problems are solved by means of DSVM. Comput...
متن کاملA moment inequality for multicategory support vector machines
The success of support vector machines in binary classification relies on the fact that hinge loss utilized in the risk minimization targets the Bayes rule. Recent research explores some extensions of this large margin based method to the multicategory case. We obtain a moment inequality for multicategory support vector machine loss minimizers with fast rate of convergence.
متن کاملMulti-category Support Vector Machines, Feature Selection and Solution Path
Abstract: Support Vector Machines (SVMs) have proven to deliver high performance. However, problems remain with respect to feature selection in multicategory classification. In this article, we propose an algorithm to compute an entire regularization solution path for adaptive feature selection via L1-norm penalized multi-category MSVM (L1MSVM). The advantages of this algorithm are three-fold. ...
متن کاملReinforced Multicategory Support Vector Machines
Support vector machines are one of the most popular machine learning methods for classification. Despite its great success, the SVM was originally designed for binary classification. Extensions to the multicategory case are important for general classification problems. In this article, we propose a new class of multicategory hinge loss functions, namely reinforced hinge loss functions. Both th...
متن کاملStructured multicategory support vector machines with analysis of variance decomposition
S The support vector machine has been a popular choice of classification method for many applications in machine learning. While it often outperforms other methods in terms of classification accuracy, the implicit nature of its solution renders the support vector machine less attractive in providing insights into the relationship between covariates and classes. Use of structured kernels c...
متن کامل